unconditional branch
Policy Gradient Guidance Enables Test Time Control
Qi, Jianing, Tang, Hao, Zhu, Zhigang
We introduce Policy Gradient Guidance (PGG), a simple extension of classifier-free guidance from diffusion models to classical policy gradient methods. PGG augments the policy gradient with an unconditional branch and interpolates conditional and unconditional branches, yielding a test-time control knob that modulates behavior without retraining. We provide a theoretical derivation showing that the additional normalization term vanishes under advantage estimation, leading to a clean guided policy gradient update. Empirically, we evaluate PGG on discrete and continuous control benchmarks. We find that conditioning dropout-central to diffusion guidance-offers gains in simple discrete tasks and low sample regimes, but dropout destabilizes continuous control. Training with modestly larger guidance ($γ>1$) consistently improves stability, sample efficiency, and controllability. Our results show that guidance, previously confined to diffusion policies, can be adapted to standard on-policy methods, opening new directions for controllable online reinforcement learning.
- North America > United States (0.14)
- Asia > Middle East > Jordan (0.04)
- Europe > Portugal > Braga > Braga (0.04)
FuzzDistill: Intelligent Fuzzing Target Selection using Compile-Time Analysis and Machine Learning
--Fuzz testing is a fundamental technique employed to identify vulnerabilities within software systems. However, the process can be protracted and resource-intensive, especially when confronted with extensive codebases. In this work, I present FuzzDistill, an approach that harnesses compile-time data and machine learning to refine fuzzing targets. By analyzing compile-time information, such as function call graphs' features, loop information, and memory operations, FuzzDistill identifies high-priority areas of the codebase that are more probable to contain vulnerabilities. I demonstrate the efficacy of my approach through experiments conducted on real-world software, demonstrating substantial reductions in testing time. Fuzz testing is a critical technique for identifying vulnerabilities in software by subjecting programs to random or semi-random inputs. As a result, large portions of the code are left unexplored, and significant vulnerabilities can go undetected.
- North America > United States > Virginia (0.05)
- Asia > Singapore > Central Region > Singapore (0.04)
- Information Technology > Security & Privacy (0.47)
- Government > Military (0.41)